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‘The case for consistent, unanibig- 
uous usage of a few key odds ex- 
pressions. 


worDs OF ESTIMATIVE PROBABILITY - 


Shernian Kent - 


The briefing officer was reporting a photo reconnaissance mission.” 


A etek Pointing to the map, he made three statements:, © 0, we 5 te St 4 
i AL. “And _at this Jacation there is a new airfield. [Ile soul have se * 
RS located it.to the second on a larger map] ‘Its longest rune *: i 
"5. way. is 10, 000 feet.” 
. 2. “Tt is almost certainly a military airfield.” 
on | ae ee 38, “The, terrain.is such. that the: Blanks -could-casily lengthen the 2.0 skin eee coe 
‘ runways, otherwise improve the facilities, and incorporate this 
' field into their systern of strategic staging bases, It is possible 
that they will.” Or, more da aringly, “Te would be logical for 
_ them: to Ao. this and sooner or later they probably will” a, Siete ce Ne ates 


the above are ‘typical of three Kinds of statements which ce - 
the literature of all substantive intelligence. The first is as close as 
one can come to a statement of indisputable fact. It describes sumo- 
thing knowable and known with a high degree of certainty. The 
reconnaissance aircraft's position was noe D Feith precision and its 
camera reproduced almost exactly what was there. 


ee ean ee eee ee 


Estimative Uncertainty 


The second is a judgment.or estimate. It describes something which 

is knowable in terms of the human understanding but not precisely 

known by the man who is talking about it. There is strong evidence 

to sustain his judgment: the only aircraft on the field are military air- 

craft, many are parked in revetted hard-stands, the support area has Bes : 

all the characteristics of similar known military installations, and so 

on. Convincing as it is, this evidence is circumstantial. It cannot 
east justify a flat assertion that this is a military - airfield. Jt makes the .. 
pte ee easer'say; 90 pereent of the way.” ‘And: sorhé: sort.cf-verbal ane alifier:’ | 


‘This particular briefing officer was not the photo-intcrpreter. See page 61 
for the special language of P/Is. 
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is necessary to show that the case is a p0-peroenter, n not a 100, This 
is why the briefer said “almost certainly,” 


The third statement is another judgment or estimate, this ¢ one made 


-almost without any evidence direct or indirect, It may be an estimate 


of something that no man alive can know, for the Blanks may not yet 
have made up their minds whether to lengthen the runways and build 


up the base. Still the logic of the situation as it appears to the briefer 
_ permits him to launch himself into the area of the literally unknowable 


and make this estimate. He can use possible to indicate that runway 


' extension is neither certain nor impossible, or he can be bolder and use 

~. probably to designate more precisely a degree of likelihood, a lower 

.., one than he had attached to his estimate eee the character of 
. the airfield: 


“Generally speaking, the most iaplortant passages of the literature of 


.. substantive intelligence contain far more statements of the estimative 
. types two and three than of the factual type one. This is the case 
_ because many of the things you most wish to know about the other 

man are the secrets of state he guards most jealously. To the extent - 
his security measures work, to that extent your knowledge must be | 


imperfect. and your statements accordingly. qualified by designators 
of your uncertainty. Simple prudence requires the qualifier in any 


-- typé-three statement to show a decent reticence before the unknowable. 


_ Concern over these qualifiers is most characteristic of that part of the | 
intelligence production business known as estimates. This is no small 


recondite compartment; it extends to almost every corner of all in- 
telligence research work, from the short appraisals or comments of 


a reports officer to the full-dress research study of the political or: 


economic analyst. Practically all substantive intelligence people con- 


ee stantly make estimates, The remarks that follow are generally ad- 


dressed to all these people and their readers, but most especially are 
they addressed to that particular institution of the estimating busi- 


- ness known as the National Intelligence Estimate and its. audience. 


_ The NIE, taking into account the high echelon of its initiators, pro- 


. ducers, and consumers, should be the community's best effort to deal 
_ , with the relevant evidence imaginatively and judiciously. It should 
» set forth the community's findings in such a way as to make clear to 


the reader what is certain knowledge and what is reasoned judgment, 
and within this large realm of judgment what varying degrees of 


. certitude lie behind each key statement. Ideally, once the commu- 
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nity has made up its mind in this matter, it should be able to choose a 
word or a phrase which quite accurately describes the degree of its 
certainty; and ideally, exactly this message should get through to 
the reader. ae 


_ It should not come as a surprise that the fact is far from the ideal, 


- that considerable’ difficulty attends both the fitting of a phrase to 


the estimators’ meaning and the extracting of that meaning by the 


consumer. Indeed, from the vantage point of almost fourteen years — 


of experience, the difficulties seem’ practically insurmountable. The 
why and wherefore of this particular area of semantics is the subject 
of this essay. oo 


Let me begin with a bit of history.? 

Early Brush with Ambiguity ; 

In March 1951 appeared NIE 29-51, “Probability of an Invasion of . 
Yugoslavia in 1951.” The following was its key judgment, made in 
the final paragraph of the Conclusions: “Although it is impossible to 
determine which course the Kremlin is likely to adopt, we believe 
that the extent of Satellite military and propaganda preparations in- 
dicates that an attack on Yugoslavia in 1951 should be considered a 
serious possibility.” (Emphasis added.) Clearly this statement is 
either of type two, a knowable thing of which our knowledge was 
very imperfect, or of type three, a thing literally unknowable for the 


_ reason that the Soviets themselves had not yet reached a binding 
_ decision. Whichever it was, our duty was to look hard at the situa- 


tion, decide how likely or unlikely an attack might be, and having 


‘ reached that décision, draft some language that would convey to the 


reader our exact judgment. 


The process of producing NIEs then was almost identical to what 
it is today. This means that a draft had been prepared in the Office 
of National Estimates on the basis of written contributions from the 
IAC § agencies, that a score or so of Soviet, Satellite, and Yugoslav 
experts from the intelligence community labored over it, and that an 
all but final text presided over by the Board of National Estimates 
had gone to the Intelligence Advisory Committee. There the IAC 


‘Harry H. Ransom’s Central Intelligence and National Security (Cambridge, 
Mass., 1958) carries on pp. 196-7 a bob-tailed and semewhat garbled version of it. 
_ ‘Intelligence Advisory Committee, USIB’s predecessor. 
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25X1A9a oon, one of the original eight members of the Board of National 
‘ ; Estimates, a lawyer by trade, and a gifted scmanticist by avocation. ‘ Some will 


. remember him for his elegant, and Precise. writing; none will forget, his Secon ities: 
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members, with the DCI in the chair, gave it its final review, revision, 
and approval. 

As is quite obvious from the sentence quoted above, Soviet and 
Satellite intentions with respect to Yugoslavia were a matter of grave 
concern in the high policy echelons of our- government. The State 
Department's Policy Planning Staff was pr obably the most important 


hope their product w ill command. 


.A few days after ‘the estimate appeared, I was. in thformal con- 
versation with the Policy Planning Staff's chairman. . We spoke of 


Yugoslavia and the estimate. Suddenly he said, “By the ays what 


did you people mean by the expression ‘serious possibility’? What 
kind of odds did you have in mind?” [told him that my personal 
estimate was on the dark side, namely that the odds were around 
65 to 35 in favor of an attack. He was somewhat jolted by this; he 
and his colleagues had read “serious. possibility” to mean odds very 
considerably awek Understandably troubled by this want of com-: 
munication, I began asking my own colleagues on the Board of National 


_ Estimates what dels they. had had in find when they agreed to that 


wording. It was another jolt to find that each Board Ree had 
had somewhat different.odds in mind and the low man was thinking 
of about 20 to 80, the high of S0 to 20.- The rest ranged in betveen, 

Of my colleagucs on the Board at least one—maybe more-—shared 
my concern. My most obvious co-worrier was] He and 
I were shaken perhaps more by the realization that Board members 
who had worked over the estimate had failed to communicate with 
each other than by the Board’s failure to communicate with its audi- 
ence. This NIE was, after all, the twenty-ninth that had appeared 
since General Smith had established the Office of National Estimates. 
Had Board members been sceming to agree on five month’s worth of 
estimative judgments with no real agreement at all? Was this the 


_ case with all others who participated—-ONE staffers and IAC repre- - 
‘sentatives, and even IAC members themselves? Were the NIEs dotted 


with “serious possibilities” and other expressions that meant very dif- 


“He ‘was ‘thé'man‘ who" always‘ wore his ‘hatin the house. ‘ 


t 


Words of Probability 


25X1A9a 


fees 


ne oh 
er 


. group | seized of the problem, Its chairman | and members.xead NIE... ee cae 
"29°51 with the sort of éoncentration: intelligence producers can’ only *” 
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ferent things to both producers and readers? ‘What were we really — 
trying to say when we wrote a sentence such as this? 
What we were trying to do was just what my Policy Planning friend. ~ j 
had assuined, namely to quote odds on this’or that being the case or 
taking place in the future. There is a language for odds; in fact 
_ there are two—the precise mathematical language of the actuary or 
the race track bookie and a less precise though useful verbal equiva- " 
lent. We did not use the numbers, however, and it appeared that” eT 
we were misusing the words, : 


ev Se Soop a Bae S08 Ae BR ane “yas ge 


‘The No-Odds Possible 


gt 


Our gross error in the Yugoslav estimate, and perhaps in its prede- 
_* cessors, lay in our not having fully understood this particular part of 
25X1A9@ our task. As M1 1 saw it the substantive stuff we had been 

. - dealing with had about ‘t certain elements of dead certainty: Stalin” 
was in charge in the USSN, fer'example. These, if relevant, we stated 
affirmatively or used impliedly as fact.’ There were also elements: of 
sheer: impossibility’ (Yugoslavia ‘was not géing to crack off along its 

= borders and disappear physically from the face of,the carth); these Pe 
“1 “we did not“bother to state at all. In between théso"matters of cers 23h 
tainty and impossibility lay the large area of the possible. With 
respect to the elements herein we could perceive some that were more - 
a likely to happen than not, some less likely. These were the elements 
upon which we could make an estimate, choosing some word or phrase aa 
td convey our judgment that the odds were such and such for: or ; 
| against something coming to pass. 
i At the race track one might say: 
Thete are ten horses in the starling gate. It is possible that any one of 
ot ; them will win—even the one with three legs. 
But the odds (or chances) against the three-legger are overwhelming. 
; Here, as in estimating Yugoslav developments, there is evidence 
to justify the citing of odds. But in the world that intelligence esti- 
mates try hardest to penetrate—a world of closed covenants secretly 
Coe. arrived at, of national business conducted behind walls of all but : 
[eens impénetrable sécurity, of skillfully planned deceptions, and-so on#3*' af . 4 
such evidence is by no means invariably at hand. In a multitude of . , 
the most important circumstances—situations you are duty bound to 
consider and report on—about all you can say is that such and such _ 
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is neither certain to happen nor is its happening an impossibility. 
"The short-and proper way out is to say that its happening is possible 
and stop there without any expression of odds. If you reserve the 
use of “possible” for this special purpose—to signal something of high 
importance whose chances of being or happenirig you cannot estimate 
with greater precision—hopefully you will alert your.reader to some 
necessary contingency planning. (You may not if you have dulled xy * bs 
pip a4 citing: a Jot of ed of little real consequence.) ; 


Beret ery ‘ > 


ag our: gross error “lay. in not perceiving the correctriess-—or: at any . Louth: oes 
pate the utility —of the above formulation, our. iy articular ‘error layin oe MORE 
-‘using the word “possibility” with the modifier “serious.” | and 29X1A9a 


I felt that it was geing to be difficult enough for the estimators to: 
_ communicate. a. sense of odds even if they stuck to a fairly rigorous’ 


Pies vocabulary; it was coing, to -be inpossibl e if the vocabul lary were 
~~ permitted to become as ee imprecise as in normal. speec eh We 
bedeee. had to have a-way of differentiating between those possible things, ee oe 
ee 4 eee! “ahout: ‘which’? awe could make™ ‘a. statement. of likelihood and. the. other” : | , 
ae ss possible things about which we could not. The frst cardinal rule to : 


“emerge was thus, “The word ‘possible’ (and its cognates oy oe not be. - 
modified.” The ur ze to drop into ordinary usage and write “Just pos- : 
sible,” “barely possible,” “a distinct [or good] possibility,” aed so on 

must be suppressed. The whole concept of “possibility” as here devel- 
oped must stand naked of verbal modifiers.*: Bl 8 ans ate 


Oe Re me on 


*See page 59. 

*This usage is wholly in accord with the findings of the lexicagraphers, who 
almost invariably assign itethe number one position. Further, it is readily under- 
stood and generally employed by statisticians, scientists, and the like, who some- 
times define it as “non-zero probability.” This is much to my taste. 

At the same time there can be no question of the existence of a second usage, 
especially in the ordinary spoken word. The meaning here is most emphatically 
not the broad range of “non-zero probability,” but a variable low order of prob- 
ability, say anywhere below 40 or 30 or 20 percent. Thus it would fall last in 
a scries that named descending odds: certain, probable, possible. When people 

“ii use. it.to.signify very low-odds, for example below 5 percent, they.may say “re- 
motely possible” or any of its many cognatcs.. This of course is not to my liking, | 
but the intended meaning is clear, The serious trouble cores when another 
group of users lifts the word out of ils position in the celldr of odds and by 
‘the addition of augmenting adjectives makes it do. duty upstairs: “serious possi- 
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-25X1AQ@Once I had deeded upon’ this first ata rule. we 


turned ¢ ments where likelihood could be estimated. We began. 

to think iu terms of a chart which would show the mathematical odds 

‘equivalent to words and phrases of probability. Our starter was a 

pretty complicated affair.” We approached its construction from the 
wrong end. Namely, we began with 11 words or phrases which 

— scemed to convey a feeling of 11 diffcrent. orders | of probability and. aa ee ee 

ee cthen: attached: numerical, odds. to- them. At: once? we' pereéived | ‘Our e =o 

folly. In ‘the first place, given the ‘exactness of ‘the intelligence _ : 


<A RR RO SOR ELLER TETD See SAT Rn BD OR ORR 92 


data we were working me the distinctions we made between one. 
set of odds and its ieMawe above and below were unjustifiably sharp. 
And second, even if in rare cases you could arrive'at such exact mathe- 
matical odds, the verbal. equivalent could not’ possibly conv OF that 
exactness. . The laudable precision would be lost on the re ader, 


Se pL ate aac ma NM Eo 


ihe, ee: So. we tried again, this time with only five’ gradations, and beginning i j oe 
. with ‘the numerical odds, -The: chart which emerged can be set down +. 07.7% 
in its classical simplicity thus: 


100% Ce rtainty 


' 


93%, So, give or take about 6% Alinost certain 


i 
i 
: 


75%, give or take about 12% Probable 


‘50%, give or take about 10% . Chances about even 


of Possibility 


30%, give or take about 10% Probably not 


The General Area. | 


7%, give or take about 5% Almost certainly not 


0% I impossibility 


Injportant note to consumers: You should be quite clear that when we say 
“such and such is unlikely” we mean that the chances of its NOT happening 
are in our ‘judgment about three to one, Another, and to you critically 
important, way of saying the same thing is that the chances of its HAPPEN- 
ING are about one in four, Thus if we were to write, “It is unlikely that ; : 
; Castro will attempt: ‘to shoot down a. U-2 betwéen, now and November, 1965,”. | a 
“we mean ‘there is in our’ view*around a 25- -percent’ chance that he will do’ an 
* a just that. Tf the estimate were to read, “Tt is almost certain Castro will 


he 


‘ . a Es not. . 4 we would mean there was still an appreciable chance, say five 
, percent or less, that he would attempt the shoot-down. - 


‘ 
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We had some charts run up che had some acouciaas in the com- 
munity. There were those who thought the concept and. the chart 
a very fine thing. A retired intelligence professional thought well: 
enough of it to put it into a book? CIA officers, addressing. “War 
Collesc audiences and the like, would sometimes flash a slide and 
talk about it. A few copies got pasted on the walls of estimates offices 
oe, in the community. Some people. were sufliciontly taken- that | ‘they oats 
+ .2,1advocated: putting it on the’ inside “back cover of every NIE asa sort 
of sure-fire handy. glossary. 
-.. There.were also those who did not think. about the idea at all, sui SOs Fa Pte! o 3 be 
Sthers m opposition to it. Some: fairly“ important people’ ie: had‘: rae 
‘a professional stake in this kind of éhinking never took the trouble. 
to learn what it was all about. A good, many did take a little trouble). -. 
land laughed: Still a third group’ found out. all they needed to know 
‘and attacked the whole proposition from a hard semantic De Spon 
OF these. more later, fee ae 


in to oe de pr 
eens . 


“iacthe dace-of this media and: gepontod and avith’ He "earls ae. 


2x1 X18 ture of my only solid ally, RL began backing away from 


Pina of: 200. bold forward: positions.. I did.continue liarassing actions and in. the’. oe cae ki 
course of making a nuisance of myself to associates and colleagues did - 


pick up some useful converts, but I dropped all thought of getting ° 
an agreed air-tight vocabulary of estimative expressions, let alone 
; -. -reproducing-the chart in the rear of every NIE. With the passage 
of time it has appeared that the guerrilla strategy thrust upon me by 
circumstance was the only one holding any chance of success. In 
almost fourteen years this article is my first serious and systematic 
attempt to get the message across, and it probably would not have 
eee written if pric ar had not consulted me about his foray 


into the same semantic problem. 


ee ee 7 


| is i 
I The Aesthetic Opposition 
{ 
/ What slowed me up in the first instance was the firm and reasoned ms 
“1. ‘yakesistance., of some of my colleagués: . Quite figuratively I am: going © | : 
Washington Tlatt, Straéegic Intelligence Praduceon (NLY., 1957). The chart 
Tae he & Appears on the inside cover and again’ on page 208-—not exactly. as above but. . 
in full accord with my principles. The trouble comes on pp. 209-210, where - 
eS a General Platt departs widely, and to me regrettably; from my notion a levitimate : 
> SyHONYINS. ° ee 
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eee . 
to call them the “poets”—as opposed to the “mathematicians”—in 
“my circle of associates, and if the term conveys a modicum of dis- 
approbation on my part, that is what I want it to do. Their attitude 
toward the problem of communication seems to be fundamentally 
defeatist. “They appear to believe the most a writer can achieve when 
working in a speculative area of human affairs is communication in 
only the broadest gencral sense. If he gets the wrong message ACTOSS 
: _ or no message at all—well, that is life. - a 
pM gt Pes 2 is knelt... +) Perhaps: F-overstate the pocts’ defeatism. In any, case at least.one 2. ee 
Jo... of them feels quite strongly that my ‘brick for: the, “mathematicians” ee 
is pretty much nonsense. He has said that my likening my sideto the. : Es 
mathematician’s is a phoney; that I am in fact one withthe sociologists ; 
i - . who try by artificial definitions to give Janguage a bogus precision. 
He has gone on to stress the function of rhetoric’ and its importance. 
‘Aud he has been at some pains ta. point out how handy it would be 
f  -. 7-1 to use” expressions. like “Just possible,” “may well’ and “doubtless” 
ee eee they are loosely used in conversation. Could there not be .an - 8, 
a aa Tae  oceasional ‘relaxation of the rule? ie ee Se, ee | : 
“2° Suppose one wrote a sentence: “Khrushchev may izell have Radin 
thie back of lis mind sueh and such, or indeed it is distincily possible 


Laan mes ee Pe 


r 


that somebady had just prined him... ” Now suppose you delete 
the “well” and the “distinctly”; has anything been lost? There will 
be those who point out that “may well” and “distinctly possible” do . 
convey a flavor which is missing without them. Of course the jlavor 
a in question is the flavor of odds, communicated without quoting them. 
i The poets would probably argue that in a sentence of this sort the 

: introduction of any of the terms for particular odds would make the 

| writer look silly. Everybody knows that you could not have the evi- 

| dence to sustain the use of, say, “probably” in these two instances. 
Hence you can only suggest odds by the use of the “may well” and 

i “distinctly possible” and so say something without saying it, in short 
ic fudge it. The poets feel wounded when urged to delete the whole s 2 
« ambiguous sentence, arguing that this serves only to impoverish the — te 
ee ; “product, . They grow, impatient when you advocate dropping, only wd ; 

"the “well” and the “distinctly.” " And as for your accusation of fudging, 


g that 


‘they generally counterattack, inviting you to ‘write somethin 
fudges nothing. 
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There is a point which the poets can make with tclling effect. It 
is that there are probalsly just as many reading poets as there are 
writing pocts, and these are going to be numb to the intended meaning 

+. of the “mathematician” writer. If you write to give no more than just . 
the general idea or gencral feel you may get through with great 
success. Per contra, if you break your heart in an endeavor to make 


nok Antena | A mm 


y t 
af eryourself fully and precisely, understood, you may NODS Oa i. ke a ee “f 4 48 
, “Trealize thé truth in the above; I am not reconciled; I deplore it. . . , 
as sig AE peas .: The Growth of Variants si se SEE ° an - 


~ Even if theré-had been, no poets it would have been an impractical | 

_; idea to print a chart on the inside of the back page.of each NIE. 
‘as a-sort of Glossary. - To have used the oné on page 55 and stuck to 
these words exclusively would have imposed intolerable restraints upon 


nua, athe prose. Even if it had been desirable-it would have beeirimpoisible | . 
‘to enforce such rigidity. But this was really never at issue: from the 
; __ Start a number of perfectly legitimate synonyms for. the concept of : : 
tase . vt. possibility and a number for each of the five orders of likelthood were | 


generally recognized.® 
*Some of these synonymous meanings are expressed in verb forms. Thus it 
is syntactically possible to use them. closely coupled to one of. the adverbial 
expressions of odds, e.g, “we believe it likely that...” or “we estimate it is 
almost certain that such and such will not...” If we really mean to assign 
an odds value to these verb forms good usage would forbid this kind of cdoubling-. 
up. Mathematically, the probabilities would have to undergo a quite ridiculous 
‘multiplication, Thus “we believe” (75-tpercent) multiplied by “likely” (75: 
percent) would yield odds vroxse than 3 to 2 instead of 3 to 1. If we are not 
assigning an odds value to “we believe” and “we estimate,” the purist would say 
we should not use them, Yet on many occasions a writer will feel uncomfortable— 
and justifiably so—with a bare “It‘is likely that . / 2” Such a bald statement 
is seemingly more confident than the situation would warrant. The writer will 


| ie nent 


feel something akin to a compulsion towards modesty and a drive to'soften the os 7 
wi. “likely” by introducing it. with a “we believe” ‘or “we estimate.”. Almost in. | i 
Det a vatiably he does nét*intend to change the odds associated with “likely.” If one : 
could set. himself up asthe arbiter, -one would, I believe, rule that the “likely,” oy Cee 
, or the “probably,” or the “almost. certainly,” ete..was the operative expression ae 
“+ of odds and that its message was unaffected by the introducing verb. | “ 
Doubling up in the “possibly” category is a different matter.. We should avoid 
_ “it might (or may) be possible for the Blanks to...” ‘The verb should, be - afoul : 
: le “, pregent, or fatura “indicative, normally.” * and: Swill bev ie aes pais ee % : 
s . 7 4 


4 


es 
eo 


a 
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For example: 


_ f ae f conceivable 
Passible 2g paddies ectkes ee dmays 


tate 


stoke nn a te ei oe 


“Almost “certain 


Achances a little: better. [or less}... 


ene 


|. . {chances about even 0 oo St Be 
than even 


improbable 
unlikely 
Probably not ...........-. we believe that... not 
we estimate that... mot : 
we doubt, doubtful ~~ : oat 
“These synonyms must rot be modified; might well, could well, just could, 
barely conceivable, etc. are as inadmissible as the original sin. aa: 


4 “Could” is included here because of many years’ duty as a synonym for 
“possible.” It has also served as a short way of noting a capability as in “The 
Soviets could develop [for “have the capability to develop”] such and such a 
radar though we have no evidence that they are doing so.” The two usages are 
close, to be sure, but not identical. : se . : hoe 


__ As in, “It is almost certain that such and such will occur in the delta, perhaps ; : 
in Saigon itself.” 

“This group of words poses at least one very vexing problem. Suppose you eh, 
casas (. «wish: to make a positive. estimate. that there -is; say,, about ‘a. 30-percent -chancé.-". * : : 
ei -": Yhat such and such’ thing is the case. Assuming that the thing in question is |. 
“ash important, a 30-percent chance of its being ‘the case is highly significant. If © 
you stick with the chart and write “it is improbable [or unlikely ete.) that such 

_and such is the case” you will probably convey a much more negative attitude 
than you intend. There are many ways around the problem; they will, however, 
require a. few more words, ; ; 


le M opewoa 


Akaka 
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{almost impossible A of a. 

ae virtually impossible i 4 

‘Almost certainly not ........ ym | 
some slight chance _ : 


highly doubtful 


If the chart were expanded ‘to take care of these, it probably would ; { 

eat: hot fit on the inside back cover of the NIE, and even if it could be made’- . *- - i : | 

“to, its complexity would probably exasperate ¢ ‘gentle reader more than ; 

it would edify him. Still worse, he would be confused by changes | 

- oe -- *. that would have to be made in it from time to. time, always to ac- : 
“commodate newcomers among the accepted expressions. “s 

The table of synonyms above did not come into being all at once; 


it has grown to its present size by , accretion. “We, believe” came in: , 
“rather early, and as l remember via General Smith himself. “We esti- 
rate” was a bit later;-“we think,” “we expect,” and “we judge” are. 
vere wre sy a upart way-in. * Tf they, make’ it all the way I trust they will. be used - 

"and understood in the * ‘probably”/“we believe” bracket. “We doubt” 
Por ae ase hag been accepted within the last few years as a legitimate equivalent. . 
; : . a of “probably not” There will be others-~I sincerely hope not very kin. 
‘many. Keeping them out will take some doing, In the past, what- 
ever the rigor insisted upen at the working and drafting level, who was 
there to tell a General Smith or a Mr. Dulles, as he presided over the - 

JAC or USIB, that the revision he had just written out on a piece Ob ae : 

yellow paper was not permissible? ; 


ene one 


Consistency in Giese 


From my remarks about the pocts, it should be clear that my sym- 
pathies lic with their mathematical opponents. But we mathemati- 
cally-inclined are ourselves not in good array. You might almost say 

that some of us are talking in the decimal, others in the binary, and 
still others in the root five or seven systems. — i 
Yor example, consider the letter-number device which has been 
standard with attaché and other reporting services, A~2, C-3, F-G, etc. __. 
‘The numbers 1, 2, 3, 4,°5; and 6 designating the quality of areport’s  * | 
content stand for, respectively: (1) confirmed by other independent 
| or reliahle sources; (2) probably true; (3)- possibly true; (4) doubt- ; ate 
: ful; (5) probably false; and (6) cannot be judged. Note that the ye : : 


Chr 


Pat 


“way in.” I toe it gets out. 


ee : * “We anticipate,” used regre ttably : as a synonym for: * “Swer expect is also. ce | 


ae poe s is ES 
ae ee a ie ack A ek celta enka 5 \ ADT een cg epee! penne net 


eo 
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. | a . : p . 2 : H 
number 3, “possibly true,” is in the middle of the scale of odds, doing ; 
the duty I have hoped it should never be asked todo,” a Pa } 

Or consider the findings of a distinguished intelligence research | ; 
project. The object was to identify certain military units with respect 
_to the chances of their existence or non-existence. One group of units 
- was called “firm,” another “highly probable,” a third “probable,” and 
a fourth general group “possible.” Except for one important thing, 
this kind of ordering was wholly to my taste. The word “firm” was ¢ 
;. fortunately not used, as one might. expect, to describe a condition... | fie 
~ oF 100 percent certainty. Its begetters, upon ‘ctoss-examination, owned. ©" 70 4 7 = 
that it was meant to indicate something like 90-95 percent—roughly - 
the equivalent of my “almost certain.” This usage puts the lower 
_ categories slightly askew from the terminology of my chart—‘highly 
probable” equating to my “probable” and “probable” to my “chances ° 

. ,.. better than even.” , “Possible,” however, was usedexactly as Thave felt | 

ft should be used, to-desigriate something in thé.range of chances be-" ++". . 8." 

tween the absolute barriers of “certainty” and “rapossibility” to swhich 

- BO numerical odds could be assigned. _ 


Se a SOA RAD TESOL RT 


» There are other heresies arnong the mathematicians, if they cari-be so '>” i a eer 
.. proclaimed. For example, look at, the way in which photo-interpreters 
have defined their key evaluative words: as" SO : 


ON mee + en 0 AOR ARO «eC RORERIL -+ 


Suspect—Evidence is insufficient to permit designation of a function with 
any degree of certainty, but photography or other information provides scme 
indications of what the function may be. : 
Possible—Evidence indicates that the designated function is reasonable . 
; and more likely than cther functions considered... : é ; : 
Probable—Evidence for the designated function is strong and other fiinc- 
tions appear quite doubtful: ; 


Pe ang tt meat He 


I This kind of formulation shows that somcone—probably a number 
: of people—had spent a good amount of time striving for a set of 
rigorous definitions. If you pause long enough to realize that the 
: photo-interpreter’s first probleny is identification and then take a hard 
‘ look at his word “suspect,” you will see that it parallels my usage for 
| “possible.” But the P/Is have preempted “possible” for other duly. : : 
'. Their “possible” fits niccly into the slot of “probable” in my. scale of -_ te 
values, and their “probable” into my “almost certain.” — fo 
We are in disarray. ahs 


Leni Wow ee i eae 


_.To Estimate or Not’ 


|. The green language of ordinary conversation abounds with estimates 
. given lightly and with a high order of confidence: “You're a shoo-in,” | 


ae 


Ps a ‘ 3 d . 
> aw r ae ee 
aay, = : : 
ape Ey Behe ee ; ay ae 
rae ig ’ ts 
oe 4 
as 
te toa 
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“Not a Chinaman’s chance,” “A million to one.” When you hear one of 
these expressions or read its more decorous counterpart you may realize 
that the mattcr at issue and the related judgment required little soul- 
. Searching on the part of the estimator. In the intelligence business, 
too, there are many occasions when the obscurities of the unknown are : 
easily pierced and we can launch an estimative “probably” or an : 
“almost certainly not” with speed and conviction, a oo | 
There are, however; -estimates’ dt the other“end. of the’spectrum=~ | 
estimates which are patently impossible to make. The green language: 
“ds equally rich in coping with these: “Search me,” “I wouldn’t have 
_the foggiest,” “Your guess is as good as mine,” and so on, | _ 
. Te is unfortunate that intelligence estimators are not allowed this kind 
"of freedom in brushing off requests for estimates of the’ totally im-" 

., _penetrable. . Some way, or. another-a convention. has. been €stablished:. -. - { 
by whieh wwe may not.write the. sentence: “Itis impossible.to. estimate... .. 6 a! 
such and such.” If we try this mancuver our masters will often rudely - | 

‘ 


Giaieseeneiakiai | Words of Probability a : 
| 

i 

i 

: 


4 


cn Ane rn tn 8 ene bel rfl in pe 


- . ask, “Why can’t: you; what are you paid for, anyway?” -If they do not' -.. . 

~ bludgeon’ is thus, they employ a combination of blackmail and flat: | 

tery before which even the most rightcous among us are likely to fall. 

io... The play “goes like this: “You ‘sty you ‘canndt.eatiniate the: riumber: 

type, and performance characteristics of Chinese Communist long- ; 
range missiles for mid-1970. This is data which is absolutely essential i 
for my planning. Obviously no one expects you-to be wholly accurate : 
or yery confident of your; findings, .But-you people are after all the’ +. 
‘experts, and it would be too bad if I had to go to others for this stuff 
who know far less about it than you. And that is exactly what I will 
do if you refuse my request.” 

At this point we do not invite our would-be consumer to seck out 
his own crystal ball team. We accept his charge, but with grave 
reservations. Sometimes we try to stay honest by introducing con- 
tingencies. “This will probably continue to be the case but only 

yo if... if..., andif....”' Then without closing out the contingencies 

Ee with firm estimates (which we are plainly unable to make) we merely 

talk about the “ifs,” hoping that he wiil keep them in mind as time 
unfolds and that when sufficient returns are in he. will himself make | 

'. the estimate or’ask us to have asccond lobk, oer joe. Sue 
At other times again, when it is the whole subject rather than one | 
of its parts that cannot be estimated, we meet the impossible frontally. 

: _ We scrupulously avoid the word “estimate” in describing the docu- | 
ment and its findings. Rather, we proclaim these to be ‘intelligence ~ | 
eSSumptons- er. planning, . Ju our, opening paragraphs ‘we ‘are likely <7s-1%- 7 


ee othe ete 
‘ 


ves +6 
Spey 

. < oe, SS ie ees 

RRL se aye eee GEA IPL ENN RO Pa me 
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: The. Lurking Weasel. aa oe, a se 


ser” 


to be quite specific as to where our evidence begins and ends, how 


“we are speculating about quantities of things that the other man may 
~ produce without knowing whether he has” yet made the decision to. 


& 
produce so many as one. We acknowledge our use of the crutch of - 


US. analogy, and so on, We promise to speak, not in discrete figures, 
but in ranges of fi figures and ranges of our uncertainty regarding them. 


Some years back we were obliged, by fore @ majeur to compose some 
tables sctting forth how the. Blanks might divide’. up an all-but- 


‘undreamed- ot stockpile of. fissionable material among an as- -yet-unborn , 


family of weapons. There were of course the appropriate passages 


: of verbal warning, and then, “on the chance that the numerical tables’ 


should hecoine physically separated from the w ‘arming, the tables were 


overprinted in red, “This table is bascd on assumptions stated in... . 


Moreover, it should not be used for any mspose whatev er without 


inclusion, in full,*of the cautionary matéiial ini...” More recently + 


we have issued a document which not, only oon with a fulsome 


caveat, but was set. off by. a sone and. color of pepe! thet were nev. <s 
, Mepartur es. 


Unhappily, ialane the oasy estimate is not - the commonplace of 
our trade; making the impossible one is happily equally rare. What 
is the cornmonplace is the Uificult but not impossible estimate. And- 
how we, along with all humanity, hate the taskl How. fertile the 
human mind in devising wavs of delaying ‘if not avoiding the moment 
of decision! How rich the spoken language in its vocabulary of issue- 
ducking! “I have a sneaker that... 
if. . "expressions with sound but upon reflection almost without 
meaning. How much conviction, for example, do you have to have 
before you become possessed of a. sneaker; how much of the unex- 
pected does it take.to cause your heart to fail? . 

Even the well-disciplined intelligence brotherhood similarly quails 
before the difficult but not impossible estimate and all too often resorts 


to an expression of avoidance drawn from a more elegant lexicon... 


What we consciously or subconsciously seek is an expression which 
conveys a definite meaning but at the same time either absolves us 


-completely of the responsibility or makes the estimate at enough re: °° - 


moves from ourselves as not to implicate us. The “serious [or distinct] 
possibility” clan of expressions is a case in point. 35 


.” “Pd drop dead of surprise a 
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‘Look at our use of “apparently” and “seemingly” and the verbal 
“appears” and “scems.” We, the writers, are not the unique beings 
_ to whom such and such “appears” of “seems” to be the case; with these 
words we have become everybody or nobody at all. .So also’ with 


_ “suggests” and “indicates.” Perhaps the “to us” is implicit, but we do 


not so state; and far more importantly, we practically never say why 


our suggestibilities were aroused or assess the weight of the reason 
that aroused them. So still again with “presumably,” “ostensibly,” 7 
and—most serious of all-—“reportedly” otherwise unmodified. The" * 
latter. taken literally and by itself carries no evalustive weight what- “"- 


-soever, anc who should know this better than we ourselves who cach 


day handle scores of “reports” whose credibility runs wp and down 
the scale between almost certain-truth and almost-certam nonsense. 
Itisa pleasure to report—-authoritatively—that you will find very few 
unmodified “reportedlys” in the NJEs. " 


We say “the Soviets ‘probably fear that. such’ and such action awill 


‘eatise-thus and'so.” What T think we mean is “Phe Soviets probably | - 
estimate that if they do such and-such the. reaction -will-be disadvan- - 


“tageous to them,” If we say “they probably hope . . .” we mean 
roughly the opposite. We talk of another country’s willingness “to: 
_visk su¢h and-such.” This is a shorthand; and. probably’ an unconscious - 


‘one, for the country’s having estimated the odds ageinst the unwanted 
thing's: happening as well as how unaceeptable the uiwanted thing 
would be if it occurred. Its “risking the danger” removes the critical 
judgment a step or two from our personal responsibility, hiya 
"Words and expressions itke these are far-too much a part of us and 
our habits of communication to be banned by fiat. No matter what 


is said of their impreciseness or of the timidity of soul that attends - 


their use, they will continue to play an importent vart in written, ex- 
4) 1 
ression. If use them we must in NIEs, let us try to use thera sparingly 
: pes 
and in places where they are least likely to obscure the thrust of our 
key eslimative passages. og Tope 
Here may I return to the group to which I have especially addressed 


the foregoing—the brotherhood of the NIE. Let us mect these key 


estimates head on. Let us isolate and seize upon exactly the dung 


that. needs: estimating... Let us endeavor to make clear to the reader 


that the passage in’ question is of critical iniportance—the gut estimate, 


_ as we call it among ourselves. Let us talk of it-in terms of odds or 


chances, and when we have made our best judgment let us assign it a 


. word or phrase that is chosen from one of the five rou th categories of 
Bot ; ; : ERNE EES, > 


sy 
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likelihood on the chart, Let the judgment be unmistakable and let 
it be unmistakably ours. * ; ie i 


SSM Ng pO, tent ee 


If the matter is important and cannot be assigned an order of likeli- 
hood, but is plainly something which is neither certain to come about 
nor impossible, Ict us use the word “possible” or one of its stand-ins— 

". and with no modifier, ~ ¢ oo ate ; oe 


. panes Ne, = . oe 
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